EUROO 

|> 



university of 
groningen 


Authentic oral language production and interaction 
in CALL: An evolving conceptual framework for the 
use of learning analytics within the SpeakApps project 


Mairead Nic Giolla Mhichil 1 , Jeroen van Engen 2 , 

Colm O Ciardubhain 3 , Gearoid O Cleircin 4 , and Christine Appel 5 


Abstract. This paper sets out to construct and present the evolving conceptual 
framework of the SpeakApps projects to consider the application of learning 
analytics to facilitate synchronous and asynchronous oral language skills within this 
CALL context. Drawing from both the CALL and wider theoretical and empirical 
literature of learner analytics, the framework sets to theorise and problematise one 
construct, such as the authenticity of language production and interaction during a 
CALL mediated session. It is considered within this paper that a CALL learning 
analytics approach is usefully conceptualised from a student-centred perspective as 
opposed to an interventionist perspective based on action-oriented approaches to 
language learning. The paper considers that structured data -usually represented 
by statistics from Virtual Learning Environments (VLEs) or Learning Management 
System (LMS)- and unstructured data (user generated content) can be harnessed to 
support the oral language learning process. The paper considers the implications of 
using this framework with the SpeakApps platform. 
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1. Introduction 

The SpeakApps project, funded by the Lifelong Learning Programme of the 
European Commission, was designed to engage with some of the challenges and 
issues faced by language learners by providing both technical and pedagogical 
solutions for teachers and learners in the project’s target languages. The emphasis 
of many online language learning environments is usually to enhance three of the 
four language skills, such as writing, reading, and listening (Appel, Santanach, & 
Jager, 2012). The SpeakApps project’s objective was to develop a bespoken online 
environment designed specifically to enhance the oral language competencies of 
its users and to support teachers by providing access to contextualised task-based 
activities and resources. The principle outcome of the project is the SpeakApps 
platform: www.speakapps.en. The platform is Moodle based and combines virtual 
classrooms with the SpeakApps tools (Langblog, Tandem and VideoChat), with 
a Mahara space to support the SpeakApps community of teachers and an Open 
Educational Resource (OER) which houses the task-based language learning 
activities designed and categorised using the Common European Reference 
Framework for Languages (Council of Europe, 2001). Furthermore, info blogs, 
technical and pedagogical user-guides and tutorials are available on the platform. 
The SpeakApps tools can also be downloaded from the platform and have been 
designed using Learning Technology Interoperability standards, thus facilitating 
their integration across a wide variety of LMS and other virtual spaces. 

The SpeakApps project is currently engaged in its second iteration and its objectives 
include the integration of five new languages into the SpeakApps platform, the 
engagement with new language education sectors, the enhancement and the design 
of a scalable digital language learning environment and the dissemination of 
SpeakApps methodologies and solutions to the wider community. The SpeakApps 
project partners are developing a research agenda based on their current work in 
the area of online oral language learning but have also expanded this to consider 
broader educational based questions such as the use and the application of learning 
analytics within this domain. This paper briefly sets out the SpeakApps conceptual 
framework and the means by which this framework is evolving to consider learning 
analytics in the context of the SpeakApps platform. 

2. Method 

The evolution of the conceptual framework underpinning the SpeakApps project 
can be considered as being aligned to Maxwell’s (2013, p. 39) considerations 
of such frameworks to include the fundamental concepts, assumptions, theories 
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and expectations which inform the research. This conceptualisation is guided not 
only by the theoretical literature but also expands to draw from the experiential 
knowledge of the researchers and the empirical findings from the research itself. 
The activities engaged in to evolve the framework include a review of the learning 
analytics and CALL literature, the completion of thought experiments by the 
researchers to consider, and the problematisation of the use of learning analytics 
to further authentic language production, interaction during a CALL mediated 
session, and the design and implementation of pilot studies to test and refine the 
framework. The following section outlines the overarching SpeakApps conceptual 
framework and the findings of the research conducted to evolve the framework 
with respect to learning analytics based on the literature review. 

3. Conceptual framework discussion 

A fundamental cornerstone of the original SpeakApps conceptual framework was 
based on the acknowledgment that developing oral language competencies is 
particularly challenging, independent of the learning environment. This assumption 
was further evidenced as part of a comprehensive need analysis of 815 learners 
within the SpeakApps project. A further belief based on the experiential knowledge 
of the project team was that providing sufficient opportunities and space for the 
learner to produce language and to receive feedback is a significant challenge 
which learners and language teachers navigate on an ongoing basis. Additionally, 
the literature emphasised that the ephemeral nature of speaking itself contrives to 
complicate the monitoring of progress by learners and teachers, as both must rely 
on the memory of what was said (Wells, 1999, p. 115). Furthermore, the range of 
opportunities available for learners to practice a language beyond the classroom 
can be quite restricted (Gardner, 2001). The status and context of use of some 
languages can impede students in their oral interaction of that language on a daily 
or indeed a regular basis, as in the case of some minority languages such as Irish. 

The provision and use of authentic material in the learning process, which is 
used to underpin meaningful language interaction opportunities, was considered 
particularly important (Brandi, 2002) where access to formal and informal 
acquisition opportunities in the target languages are limited to learners. This 
understanding was of particular relevance in the design and contextualisation of 
tasks developed and made available in the SpeakApps OER. Designing context- 
driven tasks facilitates the learner to consolidate language meaning from context 
which in turn can lead to deeper understanding (Ellis, 1995). The SpeakApps task- 
based approach incorporates the use of these materials in the target language and 
guides learners to practice a language with respect to real life experiences and 
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scenarios. The SpeakApps pedagogical framework is situated in action-oriented 
approaches to language learning where teacher and learner agency are viewed 
as critical and are at the centre of curriculum and task design (Engestrom, 2006; 
Lipponen & Kumpulainen, 2011; van Lier, 2004). 

Action-oriented approaches are drawn from sociocultural and cultural historical 
activity theoretical approaches to language teaching and learning (see for example 
Blin, 2010; Blin & Appel, 2011; Blin & Thome, 201 1). A review of the literature 
was undertaken to commence the initial expansion of the conceptual framework 
to consider the integration and use of learning analytics within the SpeakApps 
environment. The review revealed the following insights and challenges in 
considering developing learning analytics based on the SpeakApps platform. 

3.1. Learner agency, data sources, 

oral language constructs and privacy 

Fundamental to the action-oriented language learning approach is the notion 
of learner agency. Maintaining the agency of the learner was viewed as key in 
some areas of the analytics literature. Adopting a learner-centred approach to 
analytics is claimed to support inquiry-led and active learning, which implies 
that the learner reflects on the analytical data and actively determines a course of 
action. It was considered by the SpeakApps researchers that a learning analytics 
objective of being purely interventionist and prescriptive could potentially inhibit 
learner agency; the objective of learning analytics in SpeakApps should be to 
facilitate feedback and adaptation (Chatti, Dyckhoff, Schroeder, & Thus, 2012). 
The SpeakApps LMS provides a wide variety of data based on user activity. The 
analytics literature advocated the use of both structured and unstructured data in 
the development of a comprehensive analytics approach (Dawson, 2010; Siemens 
& Long, 2011). The SpeakApps platform is, however, a closed environment and 
access to unstructured data from the platform is, therefore, limited to engagement 
in Mahara. Metrics are available from the LMS and could possibly be modelled in 
a variety of ways to establish possible correlations between behavioural data and 
language proficiency as benchmarked by the CEFR. The experiential knowledge 
of the SpeakApps researchers problematised that those who engage more with 
language activities correlated to enhanced proficiency. From this basis, measuring 
online behavioural characteristics using LMS metrics may provide correlations 
between online language activities and proficiency. 

Conceptual limitations are well documented in the analytics literature related to 
such an approach and include (a) the contested notion of using (only) LMS metrics 
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as an indicator of learning, (b) the corruption of these metrics, (c) the validity 
and reliability of models and the considerations of learner characteristics, and 
(d) the quality of feedback received by learners based on such approaches (see 
Gibson, Kitto, and Willis (2014) for further discussion). Developing the platform 
to encompass a personal learning environment may provide a greater opportunity 
to include data related to students’ engagement with other online resources; this 
presents a further challenge with the integration and aggregation of multi-formatted 
data from a variety of sources (Chatti et al., 2012). 

Challenging as integrating multi-sourced data may be, there is also a specific 
need to develop a privacy and ethical framework to protect learners and to ensure 
that they are active agents and collaborators within this process. SpeakApps 
researchers, whilst exploring if the authenticity of language interaction could be 
measured using a learning analytics framework, considered the following activities 
in oral production as potential areas where learning analytics could be applied to 
provide feedback to the learner: 

• listening and responding to questions measured in the turns of interactions 
and transitions; 

• new language forms and meaning, i.e. application in a variety of contexts; 

• intonation and pronunciation. 

4. Conclusions 

The SpeakApps conceptual framework has at its centre the notion of action- 
oriented approaches to language learning. The platform in its current form 
gathers LMS behavioural data. Furthermore, the platform stores audio and video 
files created by both learners and teachers. The analysis of SpeakApps richer 
multimedia data is potentially targeted at a wider breadth of stakeholders including 
learners, teachers, institutions and researchers. The objective of learner analytics in 
SpeakApps is linked to the objective of facilitating learner agency in the language 
learning process. This paper, however, did not consider the means or techniques 
(i.e. statistics, data mining or visualisations) to analyse SpeakApps data. Initial 
considerations examine the use of pattern modelling and the use of statistics to 
underpin this analysis. SpeakApps researchers in the next phase are setting out 
to operationalise how the SpeakApps data can be collected, pre-processed and 
analysed whilst concurrently establishing and considering ethical and privacy 
issues of learners and the use of their data. Learners’ expectations with respect to 
the use of data and the principle of informed consent is central to promoting active 
and collaborative student engagement. There are significant issues associated 
with the implementation of learning analytics within a learning environment, the 
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breadth of which is beyond the scope of this paper. However, in the evolution of the 
SpeakApps conceptual framework, a consistent focus on learner agency is being 
maintained. 
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